Columbia Kermit/kermit.zip/newsgroups/misc.19970929-19971216/000228_news@newsmaster….columbia.edu _Sun Nov 2 08:55:50 1997.msg

Internet Message Format | 2020-01-01 | 17KB

Return-Path: <[e-mail]>
Received: from newsmaster.cc.columbia.edu (newsmaster.cc.columbia.edu [128.59.35.30])
    by watsun.cc.columbia.edu (8.8.5/8.8.5) with ESMTP id IAA19263
    for <[e-mail]>; Sun, 2 Nov 1997 08:55:50 -0500 (EST)
Received: (from news@localhost)
    by newsmaster.cc.columbia.edu (8.8.5/8.8.5) id IAA07878
    for kermit.misc@watsun; Sun, 2 Nov 1997 08:55:49 -0500 (EST)
Path: news.columbia.edu!sol.ctr.columbia.edu!spool.mu.edu!uwm.edu!vixen.cso.uiuc.edu!logbridge.uoregon.edu!dispose.news.demon.net!demon!news.demon.co.uk!demon!fangorn.demon.co.uk!news
From: [e-mail] (Adrian Godwin)
Newsgroups: comp.protocols.kermit.misc
Subject: Re: Problem with TRANSMIT / Linux 2.0.27 / Redhat 4.1
Date: 30 Oct 1997 01:15:01 -0000
Organization: Adrian's rest home for middle-aged electronics
Message-ID: <638n2l$[e-mail]>
References: <637i8i$n87$[e-mail]>
NNTP-Posting-Host: localhost.fangorn.demon.co.uk
X-NNTP-Posting-Host: fangorn.demon.co.uk [158.152.8.130]
Lines: 239
Xref: news.columbia.edu comp.protocols.kermit.misc:8008

In article <637i8i$n87$[e-mail]>
[e-mail] (Frank da Cruz) wrote:

> You know, of course, that the TRANSMIT command is a last resort, to be
> used when an error-checking-and-correcting protocol is not available, and
> that it is very likely to result in corrupted or missing data at the
> receiving end, right?

I do. That's why I'm using it to bootstrap a hex-encoded kermit server into
a DS5000 microcontroller that has only a hex file reader in its boot rom.


> : I suppose I can't be totally sure of the reliability of the serial
> : port, PC chipsets and UARTS being what they are.
> :
> You might have answered your own question right there.

I might, though I'm a bit surprised that I haven't seen any problems in the
other uses of the port if it's an especially unreliable one. Problems in
the driver might be another matter, since its use by PPP is probably a
lot different to its use by kermit.


> : set file type { text / bin }
> :       ( the number of characters lost varies slightly )
> :
> If you read the manual, you'll see that this also makes a difference in
> the number of characters sent.

Well, that's why I tried it : I wanted to see if some extra CRs would
change the number of characters buffered and hence affect the
behaviour.  In fact, I got a result which surprised me : because I didn't
set TRANSMIT LINEFEED ON, the number of characters sent to the driver
were actually the same (though CR was substituted for LF) BUT the
characters transmitted differed .. typically one character less of the
final line was sent in text mode.

> : enabling local echo
> :       ( the whole file is echoed to the screen, but not sent to the
> :         line. There's another effect too : echo only works in binary
> :         mode, not text )
> :
> The relevant command is "set transmit echo on".

And "set terminal echo on", and especially "set transmit prompt 0".
I used all these, but since I was trying various things out, probably
not consistently enough.  Anyway, I tried to repeat what I thought I'd
seen, using a script for consistency, and I couldn't - so I guess I was
wrong.  Sorry.
The only object of enabling echo was to find out whether the lossage
was due to the characters not being sent, or the file not being read.
It did at least prove that the file was all being read.


> : Any further suggestions gratefully received.
> :
> Read the manual?  There is a whole chapter on the TRANSMIT command and
> how to control it.  If you have accounted correctly for the

I did that (for that chapter, at least) I wouldn't dare post here without :-).

> characteristics of the thing that you are transmitting to and have given
> all the appropriate SET TRANSMIT commands, as well as the necessary
> communications-related commands -- speed, parity, flow control, etc --
> and you still have this problem, then supply all of the details and
> settings and we'll take a more detailed look.

For test purposes, there is no other unit : the serial cable has
handshaking looped back and there's just a protocol analyser to examine
the data. It's a bit old, but it has no problem with 19200 bps.

> Or you might just try cranking the connection down to a lower speed.

Speed changing was interesting. I tried initially at 19200. I got the
same, or very similiar results at 1200 and even at 300. The effect of
speed changes is more noticeable in binary mode : i.e. characters are
usually lost, but in binary mode higher speeds seem to cause wider
variations.

What did change the behaviour drastically was "transmit pause". Any
non-zero setting would completely fix the problem in binary mode, but
at the cost of severely limiting througput (probably due to a minimum
useful value for msleep on this hardware). Below 12, the problem in
text mode was not affected. At 12, the problem was fixed for text, too.

All this makes me think that the problem is due to the way buffering is
handled in the driver : if the file ends (and the tty mode is restored)
with data still buffered, some may be lost. The way to test this would
probably be to INCREASE the speed, so the UART can serialise data faster
than Kermit can send it. But I can't reliably check output above 19200.

I have read somewhere a description of a problem with flushing Linux
serial buffers, timeouts, closing files etc., but I can't find it now.
It's not in the serial 'howto', or the device driver source - if this
rings a bell with anyone, I'd like to know.

Here's a script that illustrates the problem, and all the details of
running it.  Unfortunately, it may be machine-speed dependent, I
suppose. For what it's worth, this machine is a 100MHz PCI 486. I
appreciate that some lines (like set file type text) are redundant in
the scripts as shown, but they illustrate some of the settings I've
played around with.


show fea
show ver
set line /dev/cua0
set sp 300
set flow none

set file type text
set transmit prompt 0
set transmit echo on
set terminal echo on
set file type bin
; set transmit pause 12

; If enabled, this string is sometimes partially sent
; ( just like the last line of the real file )
; set transmit eof <end-of-file>

show transmit
xmit example.hex


This was the result of executing

  'kermit -y test.ksc >test.lis'

(this build has no 'log debug'). The file dump at the beginning is the
result of enabling echo, but appears first in the output. I did try to
get a copy of the actual transmitted output by looping TXD back to RXD,
but it doesn't really make a useful trace : stdout isn't meant for this
sort of use!

On this occasion, the file was truncated 3 characters short of the end :
the last line transmitted was

<LF>:00000001

with no final FF<LF>.


-adrian




:03000000020038C3
:03000B000200CD23
:100030000D0A50726F62650075810F1200B61200D2
:10004000541200E2D2AF9000301200B19000301292
:10005000011F80FE758DFD758BFF53890F4389202D
:10006000438840759850438780D29922209807E5AD
:1000700022B401F8D322C298E599C3223099FDC277
:1000800099F59922C0E0C4118ED0E0118E22540F50
:1000900024028380E7303132333435363738394102
:1000A0004243444546740D117C740A117C22117C34
:1000B000A3E49370F922758C00758A005389F0438C
:1000C0008901D28C752164752201D2A922758AFF1B
:1000D000758CDBD28CD52109752164D5220375225C
:1000E000013285FF9074103124741031247413315F
:1000F00024740D312474803124752000740431245B
:10010000740E3124781474203127D8FA22B40A02EC
:1001100080E3B40D1274807520008008310DA3E4D3
:100120009370F922C380190520D33140E520B40A29
:100130000474C080EFB41407752000748080E52239
:1001600090316640FC22C296C297C294D294C29447
:08017000A293D29485FF9022B6
:00000001FF

Major optional features included:
 Network support (type SHOW NET for further info)
 Hardware flow control
 External XYZMODEM protocol support
 Latin-1 (West European) character-set translation
 Latin-2 (East European) character-set translation
 Cyrillic (Russian, Ukrainian, etc) character-set translation
 Hebrew character-set translation
 Kanji (Japanese) character-set translation
 REDIRECT command
 RESEND command
 Fullscreen file transfer display
 Control-character unprefixing

Major optional features not included:
 No debugging

Compiled Mar 14 1997 16:13:51, options:
 TLOG BIGBUFOK XFRCAN CK_SPEED CK_APC CK_AUTODL CK_MKDIR WHATAMI DYNAMIC
 CMDDEP=64 CKMAXPATH=1023 MAXGETPATH=128 CMDBL=4072 VNAML=64 ARRAYREFLEN=128
 FORDEPTH=10 MAXTAKE=32 MACLEVEL=64 MAC_MAX=256 MSENDMAX=100 MAXDDIR=32
 MAXDNUMS=4095 UNIX DIRENT RENAME CK_TMPDIR CK_TTYFD NETCONN TCPSOCKET
 SOL_SOCKET TDP_NODELAY RLOGCODE CONGSPD SELECT NOFILEH NOSETBUF NOKVERBS
 _POSIX_SOURCE __linux__ POSIX i386 __STDC__ __GNUC__ CK_ANSIC CK_ANSILIBS
 _POSIX_JOB_CONTROL CK_POSIX_SIG CK_CURSES CK_NEWTERM CK_WREFRESH CK_PCT_BAR
 CK_RTSCTS POSIX_CRTSCTS CK_DSYSINI CK_SYSINI CK_INI_A CK_TTGWSIZ CK_NAWS
 DCMDBUF CK_RECALL CK_TIMERS


Versions:
 C-Kermit 6.0.192, 6 Sep 96
 Numeric: 600192
 Patches:
    01 (handle-old-timestamps)
    05 (keep-packets-inside-window)
    06 (move-from-send-list-should-delete)
    08 (handle-spaces-in-command-line-filename)
    15 (make-xecho-flush-output)
    17 (linux-posix-hispeed)
    18 (fix-shell-wildcard-expansion)
 UNIX Communications support, 6.0.169, 6 Sep 96 for Linux
 UNIX File support, 6.0.115 6 Sep 96 for Linux
 C-Kermit Protocol Module 6.0.095, 6 Sep 96
 C-Kermit functions, 6.0.133, 6 Sep 96
 Command package 6.0.088, 6 Sep 96
 User Interface 6.0.177, 6 Sep 96
 Character Set Translation 6.0.024, 4 Jul 96
 CONNECT Command for UNIX, 6.0.083, 6 Sep 96
 Dial Command, 6.0.091, 6 Sep 96
 Script Command, 6.0.028, 8 Feb 96
 Network support, 6.0.078, 6 Sep 1996

 File type: binary
 See SHOW CHARACTER-SETS for character-set info
 Terminal echo: local
 Transmit EOF: none
 Transmit Fill: none
 Transmit Linefeed: off
 Transmit Prompt: none
 Transmit Echo: on
 Transmit Locking-Shift: off
 Transmit Pause: 0 milliseconds
From [e-mail]  Sun Nov  2 11:42:51 1997
Return-Path: <[e-mail]>
Received: from newsmaster.cc.columbia.edu (newsmaster.cc.columbia.edu [128.59.35.30])
    by watsun.cc.columbia.edu (8.8.5/8.8.5) with ESMTP id LAA11143
    for <[e-mail]>; Sun, 2 Nov 1997 11:42:51 -0500 (EST)
Received: (from news@localhost)
    by newsmaster.cc.columbia.edu (8.8.5/8.8.5) id LAA16096
    for kermit.misc@watsun; Sun, 2 Nov 1997 11:42:50 -0500 (EST)
Path: news.columbia.edu!watsun.cc.columbia.edu!fdc
From: [e-mail] (Frank da Cruz)
Newsgroups: comp.protocols.kermit.misc,comp.os.linux.misc
Subject: Re: Problem with TRANSMIT / Linux 2.0.27 / Redhat 4.1
Date: 2 Nov 1997 16:42:42 GMT
Organization: Columbia University
Lines: 107
Message-ID: <63iai2$icp$[e-mail]>
References: <637i8i$n87$[e-mail]> <638n2l$[e-mail]>
NNTP-Posting-Host: watsun.cc.columbia.edu
Xref: news.columbia.edu comp.protocols.kermit.misc:8009 comp.os.linux.misc:223653

In article <638n2l$[e-mail]>,
Adrian Godwin <[e-mail]> wrote:
: In article <637i8i$n87$[e-mail]>
: [e-mail] (Frank da Cruz) wrote:
:
: > You know, of course, that the TRANSMIT command is a last resort, to be
: > used when an error-checking-and-correcting protocol is not available, and
: > that it is very likely to result in corrupted or missing data at the
: > receiving end, right?
:
: I do. That's why I'm using it to bootstrap a hex-encoded kermit server into
: a DS5000 microcontroller that has only a hex file reader in its boot rom.
:
(etc etc)...  I'd be really surprised if the TRANSMIT command did not send
exactly the characters it was told to send -- otherwise I'd have heard about
it by now, in spades, and in any case this is a fairly stable chunk of code
that has barely changed at all in years -- so if transmitted characters are
not coming out the port, then the driver or the port hardware are the most
likely culprits.

There has been a lot of talk recently about Linux drivers -- e.g. the
ttyS devices versus the cu devices.  Have you tried using a different driver
for the same device?

: > : I suppose I can't be totally sure of the reliability of the serial
: > : port, PC chipsets and UARTS being what they are.
: > :
: > You might have answered your own question right there.
:
: I might, though I'm a bit surprised that I haven't seen any problems in the
: other uses of the port if it's an especially unreliable one. Problems in
: the driver might be another matter, since its use by PPP is probably a
: lot different to its use by kermit.
:
Wouldn't PPP hide errors from you by repeatedly forcing retransmission of
damaged material until it gets through correctly?

: > : enabling local echo
: > :       ( the whole file is echoed to the screen, but not sent to the
: > :         line. There's another effect too : echo only works in binary
: > :         mode, not text )
: > :
:
: > The relevant command is "set transmit echo on".
:
: And "set terminal echo on", and especially "set transmit prompt 0".
:
Note that when C-Kermit echos transmitted material, these are exactly the
same characters that it sends to the port.  So if they are not coming out
the port -- and the driver is not reporting an error -- something is badly
amiss under the hood.

: Speed changing was interesting. I tried initially at 19200. I got the
: same, or very similiar results at 1200 and even at 300. The effect of
: speed changes is more noticeable in binary mode : i.e. characters are
: usually lost, but in binary mode higher speeds seem to cause wider
: variations.
:
Hmmm...  If you look at the code (transmit() in ckuus4.c) you'll see that
completely different sections of code are used for text and binary mode.
Therefore if both of them fail for you, the chance that BOTH sections of code
are faulty is pretty slim.

Binary mode should not have to be used for transmitting an ordinary text
file, and especially not for transmitting one like this -- Intel Hex files
are just about the most transportable files on earth -- restricted safe
character set, short lines, etc.

: What did change the behaviour drastically was "transmit pause". Any
: non-zero setting would completely fix the problem in binary mode, but
: at the cost of severely limiting througput (probably due to a minimum
: useful value for msleep on this hardware). Below 12, the problem in
: text mode was not affected. At 12, the problem was fixed for text, too.
:
This is fairly clear evidence that the driver is allowing its output buffer
to be overrun.   It's a good thing C-Kermit has so many controls, eh?
Btw, TRANSMIT PAUSE applies per character in binary mode, and per line in
text mode (as you knew, or discovered).

: Here's a script that illustrates the problem, and all the details of
: running it.  Unfortunately, it may be machine-speed dependent, I
: suppose.
:
Well, as far as Kermit is concerned, the speed doesn't matter.  The same
code is being executed at any speed, and the same bytes are being sent;
it's up to the driver to actually send them, and to take care of realtime
considerations like flow control, kernel buffering, checking the UART's
ready and overflow flags, etc.  In any case, your hex file transmits
perfectly here in text mode.

: For what it's worth, this machine is a 100MHz PCI 486.
:
Beyond the driver, on any PC you need to be aware of buffered versus
unbuffered UART and interrupt conflict issues.

In any case, since this is a one-time-only bootstrapping operation, and
you seem to have a workaround, I think we can consider the case closed as
far as C-Kermit is concerned.  Unless there is something wrong amongst the
many ioctls given to the driver to condition it for transmitting; these
are done in the ttpkt() routine in ckutio.c, and have worked fine in Linux
for many years, but again, C-Kermit can't see which driver you are using
-- it can only issue the appropriate and documented API calls and expect
them to work as advertised.  However, if any Linux expert wants to take a
look and suggest corrections, perhaps based on changes in the APIs that
I don't know about, feel free!

- Frank